EXAMPLES OF DOMAINS 
WITH NON-COMPACT AUTOMORPHISM GROUPS 
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0^ ' Abstract. We give an example of a bounded, pseudoconvex, circular domain in C" 

for any n > 3 with smooth real-analytic boundary and non-compact automorphism 
' group, which is not biholomorphically equivalent to any Reinhardt domain. We also 

give an analogous example in C'^, where the domain is bounded, non-pseudoconvex, 
is not equivalent to any Reinhardt domain, and the boundary is smooth real-analytic 



0^ . at all points except one. 

> 

Let D he a bounded or, more generally, a hyperbolic domain in C"^. Denote by 
1^ I Aut(-D) the group of biholomorphic self-mappings of D. The group Aut{D), with 
' the topology given by uniform convergence on compact subsets of D, is in fact a 
Lie group [Kob]. 

A domain D is called Reinhardt if the standard action of the n-dimensional torus 
T'^ on 



Zj ^— > e^'^^Zj, G M, j = 1, . . ., n, 



O 

psj ■ leaves D invariant. For certain classes of domains with non-compact automorphism 
. groups, Reinhardt domains serve as standard models up to biholomorphic equiva- 
§ : lence (see e.g. [R], [W], [BP], [GKl], [Kod]). 

^ I It is an intriguing question whether any domain in C" with non-compact auto- 

^ ■ morphism group and satisfying some natural geometric conditions is biholomorphi- 

■ cally equivalent to a Reinhardt domain. The history of the study of domains with 
non-compact automorphism groups shows that there were expectations that the 
answer to this question would be positive (see [Kra] ) . In this note we give examples 

k>( ; that show that the answer is in fact negative. 

■ While the domain that we shall consider in Theorem 1 below has already been 
noted in the literature [BP], it has never been proved that this domain is not 
biholomorphically equivalent to a Reinhardt domain. Note that this domain is 
circular, i.e. it is invariant under the special rotations 

Zj ^^ e"^Zj, G M, j = 1, . . . , n. 

Our first result is the following 

Theorem 1. There exists a bounded, pseudoconvex, circular domain O C with 
smooth real-analytic boundary and non-compact automorphism group, which is not 
biholomorphically equivalent to any Reinhardt domain. 

Proof. Consider the domain 

O = {|2i|2 + \Z2\^ + l^al^ + iz^Z3 + Z^Z2)^ < 1}. 
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The domain Q is invariant under the action of the two-dimensional torus 

Zj ^ e'^^^Zj, (f)2 e K, j = 2, 3, 

and therefore is circular. It is also a pseudoconvex, bounded domain with smooth 
real-analytic boundary. The automorphism group Aut(f2) is non-compact since it 
contains the following subgroup 



Z3 



zi — a 
1 — azi 
{l-\a\^'^Z2 
lz\ 

(l-|ap)^^3 
(1 — azi)^ 



^2 ^ ,^ - ^l ' (1) 
(1 — azi) 2 ^ ' 



for a complex parameter a with \a\ < 1. 

We are now going to explicitly determine Aut(n). Let F — {fi, f2, fs) be an 
automorphism of Q. Then, since Q is bounded, pseudoconvex and has real-analytic 
boundary, F extends smoothly to ft [BL]. Therefore, F must preserve the rank of 
the Levi form Coniq) of dO. at every q G 90. The only points where Con = are 
those of the form (e*",0,0), a G M. These points must be preserved by F. This 
observation implies that /j(e*",0,0) = for all o: G M, j = 2,3. Restricting /2, 
/s to the unit disc Q, fl {z2 = z^ = 0}, we see that fj{zi, 0, 0) = for all l^il < 1, 
j = 2,3. Therefore, F{0) = (6,0,0) for some |6| < 1. Taking the composition of 
F and the automorphism G of the form (1) with a = 6, we find that the mapping 
G o F preserves the origin. Since O is circular, it follows from a theorem of H. 
Cartan [C] that G o F must be linear. Therefore, any automorphism of O is the 
composition of a linear automorphism and an automorphism of the form (1). 

The above argument also shows that any linear automorphism of Q. can be writ- 
ten as 

zi ^ e'^f^zi, 
Z2 ^ az2 + bzs, 
Z3 ^ CZ3 + dzs, 

where (pi G M, a,b,c,d G C, and the transformation in the variables {z2,zs) is 
an automorphism of the section Q n {zi — 0}. Further, since the only points of 
dO, where rankCgQ — 1 are those of the form {zi,w,±w) with w ^ and since 
automorphisms of Q preserve such points, it follows that any linear automorphism 
of 0, is in fact given by 

zi ^ e^'^i^i, 
Z2^e''l'^z,^2), 

where 0i, </)2 £ R, and cr is a permutation of the set {2, 3}. 

The preceding description of Aut(f2) implies that dimAut(f2) = 4. That is to 
say, each of the four connected components of Aut(O) is parametrized by the point 
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Suppose now that Q is biholomorphically equivalent to a Reinhardt domain 
D C C^. Since Q is bounded, it follows that D is hyperbolic. It follows from [Kru] 
that any hyperbolic Reinhardt domain G C C"' can be biholomorphically mapped 
onto its normilized form G for which the identity component Auto((j) of Aut((j) is 
described as follows. There exist integers 0<s<t<p<n and Ui > 1, i — 1, . . . ,p, 
with Yl^=i — ''^i real numbers a^, z = 1, . . . , s, k = t + 1, . . . ,p, and j = 
s + l,...,t,k = t + l,.. . ,p such that if we set = {zm+.-.+m-i+i, ■ ■ ■ , Zm+-+ni) , 
i = 1, . . . ,p, then Auto(G) is given by the mappings 



Z I ^ : : jr- , t — 1 , . . . , 5, 

Z-' Z-' + , j = s + l,...,t, 

, ,n-=.+iexp(-/3|(2^V.^ + |e^f)).'= 
z^C" ^- ^ -J- ^ — , k = t + l,...,p, 

where 

e U{nj), e C"^ j = s + 1, . . . , t, 
eU{nk), k = t+l,...,p. 

The normalized form G is written as 
G=l \z^\ < < 1, 



(2) 



(3) 



zP 



nu (i-i-f)"^n*=.+iexp (-/3fi.^f) 

where Gi :— Gf] {2;* = 0, i = 1, . . . , t} is a hyperbolic Reinhardt domain in C"'+^ x 
••• X C"f. 

It is now easy to see that, for any hyperbolic Reinhardt domain D C written 
in a normilized form D, Auto(i^) given by formulas (2) cannot have dimension 
equal to 4. 

This completes the proof. ■ 



Remark. The theorem can be easily extended to C"^ for any n > 3 (just replace 
\zi\'^ in the defining function of Q by Yl]j=i kjP; ^2 by Zn-i, zs by z^). 

There is considerable evidence that, in complex dimension two, an example such 
as that constructed in Theorem 1 does not exist. Certainly the example provided 
above depends on the decoupling, in the domain O, of the variables Z2, Zs from the 
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The work of Bedford and Pinchuk (see [BP] and references therein) suggests that 



where a is a positive integer. It is also a plausible conjecture that any bounded 
domain in with non-compact automorphism group and a boundary of finite 
smoothness C'^ for A; > 1, is biholomorphically equivalent to some Clai where a >1 
and is not necessarily an integer. Of course all the domains fla are pseudoconvex 
and Reinhard. 

However, as the following theorem shows, if we allow the boundary to be non- 
smooth at just one point, then the domain may be non-pseudo convex and be non- 
equivalent to any Reinhardt domain. 

Theorem 2. There exists a bounded, non-pseudoconvex domain C with non- 
compact automorphism group such that dQ is smooth real-analytic everywhere ex- 
cept one point (this exceptional point is an orbit accumulation point for the auto- 
morphism group action), and such that Q is not biholomorphically equivalent to any 
Reinhardt domain. 

For the proof of Theorem 2, we first need the following lemma. 

Lemma A. If Q, C is a bounded, non-pseudoconvex, simply- connected domain 
such that the identity component AutoiVt) of the automorphism group Aut{Q) is 
non- compact, then O is not biholomorphically equivalent to any Reinhardt dom,a,in. 

Proof of Lemma A. Suppose that O is biholomorphically equivalent to a Rein- 
hardt domain D. Since Vt is bounded, it follows that D is hyperbolic. Also, since 
Auto(fi) is non-compact, then so is Auto(-D). We are now going to show that any 
such domain D is either pseudoconvex, or not simply-connected, or cannot be bi- 
holomorphically equivalent to a bounded domain. This result clearly implies the 
lemma. 

We can now assume that the domain D is written in its normalized form D as 
in (3), and Auto(-D) is given by formulas (2). Then, since Auto(-D) is non-compact, 
it must be that t > 0. Next, if p = t, then D is either non-hyperbolic (for s < t), 
or (for s = t) is the unit ball or the unit polydisc and therefore is pseudoconvex. 
Thus we can assume that t = 1, p = 2, ?ii = ?i2 = 1. 

Let -Di C C be the hyperbolic Reinhardt domain analogous to Gi that was 
defined above (see (3)). Clearly, there are the following possibilities for Di: 

(i) I)i = {0 < 1^21 < i?}, < i? < oo; 

(ii) Di = {r < \Z2\ < R},0<r < R<oo; 

(iii) Di = {\z2\ < R}, < R < oo. 

For the cases (i), (ii), D is always not simply-connected, and therefore we will 
concentrate on the case (iii). If s = 0, then D is not hyperbolic since it contains 
the complex line {z2 = 0}. Thus we can assume that s = 1. Next observe that, for 
af > 0, the domain D is always pseudoconvex. Thus we may take < 0. Then 
the domain D has the form 





R 




7 > 0. 
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We will now show that the above domain D cannot be biholomorphically equivalent 
to a bounded domain. More precisely, we will show that any bounded holomorphic 
function on D is independent of Z2. 

Let f{zi,Z2) be holomorphic on D and |/| < M for some M > 0. For every p 
such that IpI < ^, the disc Ap = \ < 1, Z2 = p} is contained in D. We will show 
that df/dz2 = on every such Ap, which implies that df/dz2 = everywhere in 
D. 

Fix a point p) G Ap and restrict / to the disc AJ[^ = {zi = p,, \z2\ < Rfi}, 
where i?^ = R/2{1 - . Clearly, {/i, p) e A'^ and C D. By the Cauchy 

Integral Formula 



for I-S2I < -R/i) and therefore 



dzo 



1 

27H 



1 
27H 



dA' 



C-Z2 



dA' 



(c - pr 



Hence 



df 



dz-. 



< 



MR, 



iR,-\p\r' 

Letting |//| — 1 and taking into account that R^ — > oo, we see that \df/dz2{p>, p)\ 
as \p,\ — > 1. Therefore, df/dz2 = on A^ 
The lemma is proved. 



Proof of Theorem 2. We will now present a domain that satisfies the conditions of 
the lemma. Set 

Z'2 3 \Z2\^ . Zf 



n = <^ \zi\' + \z2r + 8\zi - 1\' - TT^-^ + =^ < 1 

\zi — 1 2 l^i — 1| zi — 1 J 

The domain Q is plainly bounded since the third term on the left is non-negative. 
Next, the identity component Auto(n) of its automorphism group is non-compact 
since it contains the subgroup 



Zl 



Z2 



Zl — a 



1 — azi ' 
(1 — azi)2 



where a E ( — 1, 1). 

Further, Q is simply-connected, since the family of mappings Ft-{zi, Z2) = {zi,tz2), 
< T < 1, retracts Q inside itself, as r — > 0, to the unit disc {|2;i| < 1, 2:2 = 0} 
(which is simply-connected). 

To show that O is not pseudoconvex, consider its unbounded realization. Namely, 
under the mapping 

2:1 + 1 

Zl ^ r, 

Zl — 1 

Z2 ^ , 
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the domain fl is transformed into the domain 

2 



n' = <(Re^i + ^|z2|^ + 2(^i--|z2|^+^^) <o 



3 

It is easy to see that at the boundary point (— |, 1) e dQ' the Levi form of dQ' is 
equal to — |-S2p5 and thus is negative-definite. Therefore, fl is non-pseudoconvex. 

Hence, by Lemma A, f2 is not biholomorphicaUy equivalent to any Reinhardt 
domain. 

Next, if (/) denotes the defining function of Q, the following holds at every bound- 
ary point of Q except (1,0): 

d(f) ^ 1 + 1-Z- 

dz\ ^1 — 1 \ 2 dz2 
and therefore grad^ does not vanish at every such point. Hence, dO, is smooth 
real-analytic everywhere except at (1,0). 

The theorem is proved. ■ 



Remarks. 



1. The hypothesis of simple connectivity in Lemma A is automatically satisfied if, 
for example, the boundary of the domain is locally variety-free and smooth near 
some orbit accumulation point for the automorphism group of the domain (see e.g. 
[GK2]). For a smoothly boun ded domain it would follow from a conjecture of 
Greene/Krantz [GK3]. 

2. Tedious calculations show that the boundary of the domain Q, in Theorem 2 is 
quite pathological near the exceptional point (1,0). It is not Lipschitz-smooth of 
any positive degree. It would be interesting to know whether there is an example 
with Lipschitz-1 boundary at the bad point. 

In fact, many more examples similar to that in Theorem 2 can be constructed 
in the following way. Let 

^' = {{zuZ2) e : Kezi+P{z2) < 0}, (5) 

where P — |^2p"^ + <5(^2) is a homogeneous non-plurisubharmonic polynomial, m is 
a positive integer, and Q{z2) is positive away from the origin. Then, by a mapping 
analogous to (4), Vt' can be transformed into a bounded domain 12. The domain Q, is 
simply-connected, non-pseudoconvex. Auto (12) is non-compact, and 90 is smooth 
real-analytic everywhere except at the point (1,0). For all such examples, dVt is 
not Lipschitz-smooth of any positive degree at (1, 0). 

It is also worth noting that, in the example contained in Theorem 2, the point 
(— 1, 0) is also an orbit accumulation point, but dO. is smooth real-analytic at this 
point. 

3. It is conceivable that the domain O as in Theorem 2 has an alternative, smoothly 
bounded realization, but it looks plausible that if in formula (5) we allow P{z2) to 
be an arbitrary homogeneous polynomial positive away from the origin with no 
harmonic term, then domain (5) does not have a bounded realization with C^- 
smooth boundary, unl ess P{z2) = c\z2\'^'^, where c > and m is a positive integer. 

This work was completed while the second author was an Alexander von Hum- 
boldt Fellow at the University of Wuppertal. Research at MSRI by the third author 

J- — ] ;„ J- u., MCTTi J- T^ATC nnool An 
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